Systems and methods for using eye signals with secure mobile communications

ABSTRACT

Apparatus, systems, and methods are provided for secure mobile communications (SMC) by an individual using biometric signals and identification in real time. The apparatus includes a wearable computing device where identification of the user is based on iris recognition, and/or other physiological and anatomical measures. Biometric identity measures can be combined with other security-based information such as passwords, date/time stamps, and device identification. Identity verification can be embedded within information that is transmitted from the device and/or to determine appropriate security measures. SMC addresses security issues associated with the transmission of eye-signal control and biometric identification data using secure interfaces with network devices within a system of systems (SoS) software architecture.

RELATED APPLICATION DATA

The present application claims benefit of co-pending provisionalapplication Ser. No. 61/991,435, filed May 9, 2014, 62/023,940, filedJul. 13, 2014, 62/027,774, filed Jul. 22, 2014, 62/027,777, filed Jul.22, 2014, 62/038,984, filed Aug. 19, 2014, 62/039,001, filed Aug. 19,2014, 62/046,072, filed Sep. 4, 2014, 62/074,920, filed Nov. 4, 2014,and 62/074,927, filed Nov. 4, 2014, the entire disclosures of which areexpressly incorporated by reference herein.

COPYRIGHT NOTICE

Contained herein is material that is subject to copyright protection.The copyright owner has no objection to the facsimile reproduction byanyone of the patent document or the patent disclosure, as it appears inthe United States Patent and Trademark Office patent file or records,but otherwise reserves all rights to the copyright whatsoever. Thefollowing notice applies to the software, screenshots and data asdescribed below and in the drawings hereto and All Rights Reserved.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to portable or wearable biometric baseuser identification and authentication for secure distributed andinteractive systems and services.

BACKGROUND

The widespread use of the internet and computing/communications deviceshas led to an explosive growth in the electronic dissemination ofinformation. However, verifiable control over the recipient(s) of secureinformation remains an important issue in the field of cyber security.Moreover, recipients of information can also become sources of sensitiveinformation where real time knowledge of the identity of such a sourcecan be an important security issue. An example of this situation isknowledge of the identity of an individual entering credit card (orother account) information during the process of making an onlinepurchase. Present-day techniques commonly used to remotely identify therecipients or sources of secure information are readily susceptible todeception. In the United States, identity theft affects approximatelyfifteen million individuals each year with an estimated financial impactof $50 billion.

To solve these problems, there is a need today to re-think systemarchitectures and roles with a specific view on data security andnon-repudiation of a user's electronic signature (e.g. password), wherethe authenticity of the signature is being challenged. In a generalsense, nonrepudiation involves associating actions or changes to aunique individual. For a secure area, for example, it may be desirableto implement a key card access system. Non-repudiation would be violatedif it were not also a strictly enforced policy to prohibit sharing ofthe key cards and to immediately report lost or stolen cards. Otherwisedetermining who performed the action of opening the door cannot betrivially determined.

Similarly, for computer accounts, the individual owner of the accountmust not allow others to use that account, especially, for instance, bygiving away their account's password, and a policy should be implementedto enforce this. This prevents the owner of the account from denyingactions performed by the account.

The ecosystem is dynamic and rapidly changing, where wireless capabilityis growing exponentially. Cloud based architectures are becoming moreappealing and attainable at manageable costs. The place to startre-thinking is with transitional and end architectures in mind and theplacement of what is called a “data abstraction layer.” This abstractionlayer is distinguished by data movement defined as data on the move anddata at rest; and includes considerations for data generation, datastorage, data processing, and the role of the server and browser in thecloud.

A first transitional step on the way to the Internet of Things (IoT) isthe emergence of fog computing or fog networking. This is anarchitecture that uses one smart device, a collaborative multitude ofsmart devices or near-user edge devices to carry out a substantialamount of processing and storage (rather than stored primarily in clouddata centers), communication (rather than routed over the internetbackbone), and control, configuration, measurement, and managementrather than controlled primarily by network gateways such as those inthe Long Term Evolution (LTE) such as 4-G LTE.

Fog Networking consists of a control plane and a data plane. Forexample, on the data plane, fog computing enables computing services toreside at the edge of the network as opposed to servers in adata-center. Compared to cloud computing, fog computing emphasizesproximity to end-users and client objectives, resulting in superioruser-experience and redundancy in case of failure. Fog Networkingsupports the IoT, in which most of the devices that are used on a dailybasis will be connected to each other. Examples include phones, wearablehealth monitoring devices, connected vehicle, and augmented realityusing devices such as the Google Glass. The ultimate goal of the IoT isto realize connections between objects, objects and persons, all things,and networks for the secure identification, management, and control ofdata.

With the above in mind, wearable display devices will challengetraditional computer human machine interaction. Today, computer mice,joysticks, and other manual tracking devices are ubiquitous tools forspecifying positional information during human-machine interactions(HMIs). With the advent of wearable computing, such bulky and obtrusivedevices that, for example, generally require stationary surfaces forproper operation are incompatible with the portable nature ofapparatuses that are designed to be worn on the body.

Wearable display devices include virtual reality (“VR”) displays such asthose manufactured by Sony, Samsung, Oculus, Carl Zeiss; head mounteddisplays (“HMDs”) such as those produced by Google (e.g., Glass®) andVuzix; augmented reality (“AR”) displays such as those manufactured byMicrosoft, Vuzix, and DigiLens; and similar devices. Eye tracking can beused to view such displays and to specify positional information.However, the eyes are also used extensively during normal humanactivities.

In a further discussion of an embodiment with respect to security, datacollected from the face, eye(s), or voice constitute unique biometricdata of the user or user groups, if desired. These collected data can beused to generate a unique private key in a system of public key andprivate key cryptography. Cryptographic systems have been widely usedfor information protection, authentication, and access control for manyyears and as such are well known in the art of information security.

An additional component for the continuous exchange of secureinformation with a biometrically identified individual is the encryptionof all transmitted (i.e., sent or received) data. Data encryption has along history that pre-dates the electronic computer. A number ofwell-established methods have been developed to protect theconfidentiality, integrity, and authenticity of data. Most encryptiontechniques make use of one or more secret keys or security codes thatcan be used to encrypt and/or decipher data streams. Keys used to encodeor decipher data streams can originate from a number of sourcesincluding previously transmitted data sequences, identification codesembedded during the manufacture of a device, and usage counts.

Encryption and deciphering methods that make use of transposition,substitution, repositioning, masking, translation tables, and/orpre-defined numeric sequences are well-known in the art. Moresophisticated techniques utilize multiple methods applied to largerblocks (i.e., more than a single character or byte) of information. Inaddition, encryption and deciphering methods that include a processingstep within a protected hardware component are generally more protectedfrom attempts at decoding compared to those implemented using softwarestored on some form of memory device.

Field-programmable gate arrays (FPGAs) and application specificintegrated circuits (ASICs) are particularly useful as encrypting anddeciphering components. In fact, Streicher et al (U.S. Pat. No.8,363,833) and others describe processes whereby even the bit streamused to program an FPGA that can be used for encryption, is itselfencrypted. Concealing both security keys and methods to decipher secureinformation greatly reduces the risk of anyone other than the intendedrecipient gaining meaningful access to an encrypted data stream.

As further background, Bluetooth had its origins in 1998 with therelease of the 1.0 specification, with a subsequent release in 2000 ofwhat was called 1.0b. These early releases were designed to remove wiresfrom the desktop of a user; these included considerations for serial,headset, cordless phone, and LAN connections. However, these earlyversions had many problems and manufacturers had difficulty making theirproducts interoperable. Subsequent releases of Bluetooth 1.1, 1.2, and2.0 included expanded bandwidth, profile capability, and finally, inrelease 2.1, new levels of security, including what is now called SecureSimple Pairing (SSP).

SSP allows two devices to establish a link key based on a Diffie-Hellmankey agreement and supports four methods to authenticate the keyagreement. One of these methods, called the Passkey Entry method, uses aPIN (i.e., personal identification number) entered on one or bothdevices. However, the Passkey Entry method has been shown to leak thisPIN to any attacker eavesdropping on the first part of the pairingprocess. If the attacker can prevent the pairing process to successfullycomplete and the user uses the same PIN twice (or a fixed PIN is used),the attacker can mount a man-in-the-middle attack on a new run of thepairing process.

Today there are numerous papers on the security risks even the mostsophisticated protocols are challenged with. New systems and methods areneeded to ensure what is termed nonrepudiation; “nonrepudiation”generally means an assurance that someone cannot deny something. In thiscase, the denial of the ability to ensure the use of a device orcommunications from a device cannot deny the authenticity of theirsignature on any use, communications, or messages that a qualified useroriginates.

SUMMARY

Although the best understanding of the present invention will be hadfrom a thorough reading of the specification and claims presented below,this summary is provided in order to acquaint the reader with some ofthe new and useful features of the present invention. Of course, thissummary is not intended to be a complete litany of all of the featuresof the present invention, nor is it intended in any way to limit thebreadth of the claims, which are presented at the end of the detaileddescription of this application.

In this disclosure, voluntary eye movements that are intended tointeract with a computing device are referred to as “eye signals.” Eyesignal controls are described in Systems and Methods forBiomechanically-based Eye Signals for Interacting with Real and VirtualObjects [Attorney Docket No. EYE-023], application Ser. No. 14/______,filed May 8, 2015, the entire disclosure of which is expresslyincorporated by reference herein.

Processes for identification of a device user are described in Systemsand Methods for Discerning Eye Signals and Continuous BiometricIdentification [Attorney Docket No. EYE-024], application Ser. No.14/______, filed May 8, 2015, the entire disclosure of which isexpressly incorporated by reference herein.

In accordance with one embodiment, apparatus, systems, and methods areprovided for a head-mounted device (HMD) that includes at least oneprocessor connected to at least one imager, where at least one of theimagers is oriented toward the eye(s) of a user. The processor isconfigured to at least one of substantially continuously,simultaneously, and/or periodically determine eye signal estimation,where the eye signal estimation is determined by the processor using theimager to detect at least one glint from the surface of the eye that hasbeen generated from a light source attached to the HMD or where theimager detects one or more distinctive features of the eye; anddetermine biometric data of a user including facial features, voice, oriris data of a user, where the biometric data is used for theidentification and authentication of the user for access and control ofat least one of the HMD, a connected device, a wireless device, and aremote server.

In accordance with another embodiment, apparatus, systems, and methodsare provided that may substantially continuously, periodically, and/oron demand perform iris recognition utilizing a head-mounted device.Biometric identification during the formation of eye-signal controls maybe used within a wide range of applications in which user identificationand/or authentication are required in real time.

In another embodiment, systems and methods are disclosed in whicheye-signal control sequences are used for authentication of a user forat least one of withdrawing money from an automated teller machine (ATM)and making online purchases.

Another embodiment discloses systems and methods to authenticate a userfor online activities including at least one of private, group, or othertesting, complying with performance requirements coupled with identityfor various forms of employment such as professional driving, piloting,or other transportation, logging hours, confirming acknowledgement toinformed consent provided orally or read by a user whereupon continuousconfirmation of identity during saccadic and other eye-based movementsduring reading, and confirming acknowledgement of any legally bindingagreement.

Another embodiment discloses systems and methods for combiningidentifying characteristics with other security tokens including atleast one of information tokens (passwords), physical tokens (keys),produced tokens (speech, gestures, writing), and other biometric tokenssuch as fingerprint and voiceprint.

Another embodiment discloses systems and methods that describe aplurality of system configurations, including:

-   -   Storing multiple user codes where searching and matching is        performed entirely on an HMD.    -   Sending user code(s) to a specific processor for identification        and matching.    -   Sending user code(s) to the cloud.    -   Augmenting or replacing common password-based access to        computing devices.    -   Substantially continuously re-verify the identity of the device        wearer.

Another embodiment discloses systems and methods for capturing an imageof the iris, or a locally generated irisCode (e.g., as described inSystems and Methods for Discerning Eye Signals and Continuous BiometricIdentification, filed May 8, 2015) from the iris image, and transmit theiris information to the cloud for authentication of any HMD.

Another embodiment discloses systems and methods that replace or augmentcommon password-based access to computing devices.

Another embodiment discloses systems and methods to use a specificimplementation of the continuous biometric identification (CBID)approach (e.g., as described in Systems and Methods for Discerning EyeSignals and Continuous Biometric Identification, filed May 8, 2015) to“buy at the aisle” by using eye-signal methods or processes referred toas “look to buy.”

Another embodiment discloses systems and methods for displaying (on anHMD or remote display device) information, including cost, about theitem.

Another embodiment discloses systems and methods for object recognitionused to identify items for purchase that are simply viewed within theenvironment of the user.

Another embodiment discloses systems and methods that establish a trueidentity of a user wearing the HMD.

Another embodiment discloses systems and methods that prevent useridentity fraud and identity theft.

Another embodiment discloses systems and methods that use the HMD toauthenticate users for at least one of educational or legal purposes.

Another embodiment discloses systems and methods that use the HMD forthe purpose of authenticating a purchase, where the authenticatedpurchase is for online purchase security and offline purchase security,where offline includes at a retail establishment.

Another embodiment discloses systems and methods that use the HMD thatincludes a second imager connected to the processor oriented outwardfrom the HMD, where the second imager detects a code that can be decodedby the processor, where the code is one of a bar code and a QR (i.e.,quick response) code and where the processor decoded data representsinformation about a product.

Another embodiment discloses systems and methods that use theinformation related to a product to allow an authenticated user tosecurely purchase the product.

Another embodiment discloses systems and methods that allow an entity toinitiate a secure communication channel with another entity by mutualgaze where the security of the communication channel may be establishedprior to communication and may be revalidated continuously or atintervals during communication.

Another embodiment discloses systems and methods that enable a secureprotocol for coordination among parties to cause an action to occurwhereupon each party performs some action during which time theiridentities are substantially continuously verified with CBID.

Another embodiment discloses systems and methods that increase securitywhen using a HMD device by limiting access to functional blocks in asilicon chip that supports eye-tracking for the HMD device.

Another embodiment discloses systems and methods that manage,coordinate, filter, and/or sequence the stimulus provided by one or morewearable devices associated with the identity of a user.

In another embodiment, systems and methods are provided that include adynamically evolving cognitive architecture for a system based oninterpreting the gaze-based intent of a user. Natural eye movement isinterpreted by the system, and used for real-time image services. Anillumination device comprising a processor and a camera is worn by theuser, with the camera aimed toward an eye of the user. The systemincludes memory with stored instructions. When the instructions areexecuted the system receives eye measurement data from the camera aimedat the user's eye. The data is used to determine a first state of theeye, and compare it to data captured from a second state of the eye.When the system determines that the first and second states of the eyeare the same, further instructions are sent to at least one processor inthe system.

So as to reduce the complexity and length of the Detailed Specification,and to fully establish the state of the art in certain areas oftechnology, Applicant(s) herein expressly incorporate(s) by referenceall of the following materials identified in each numbered paragraphbelow. The incorporated materials are not necessarily “prior art” andApplicant(s) expressly reserve(s) the right to swear behind any of theincorporated materials.

Applicant(s) believe(s) that the material incorporated by referenceherein is “non-essential” in accordance with 37 CFR 1.57, because it isreferred to for purposes of indicating the background of the systems andmethods herein. However, if the Examiner believes that any of theabove-incorporated material constitutes “essential material” within themeaning of 37 CFR 1.57(c)(1)-(3), applicant(s) will amend thespecification to expressly recite the essential material that isincorporated by reference as allowed by the applicable rules.

The inventors are also aware of the normal precepts of English grammar.Thus, if a noun, term, or phrase is intended to be furthercharacterized, specified, or narrowed in some way, then such noun, term,or phrase will expressly include additional adjectives, descriptiveterms, or other modifiers in accordance with the normal precepts ofEnglish grammar. Absent the use of such adjectives, descriptive terms,or modifiers, it is the intent that such nouns, terms, or phrases begiven their plain, and ordinary English meaning to those skilled in theapplicable arts as set forth above.

Further, the inventors are fully informed of the standards andapplication of the special provisions of 35 U.S.C. §112, ¶6. Thus, theuse of the words “function,” “means” or “step” in the DetailedDescription or Description of the Drawings or claims is not intended tosomehow indicate a desire to invoke the special provisions of 35 U.S.C.§112, ¶6, to define terms or features herein. To the contrary, if theprovisions of 35 U.S.C. §112, ¶6 are sought to be invoked to definefeatures of the claims, the claims will specifically and expressly statethe exact phrases “means for” or “step for”, and will also recite theword “function” (i.e., will state “means for performing the function of[insert function]”), without also reciting in such phrases anystructure, material or act in support of the function. Thus, even whenthe claims recite a “means for performing the function of . . . ” or“step for performing the function of . . . ”, if the claims also reciteany structure, material or acts in support of that means or step, orthat perform the recited function, then it is the clear intention of theinventors not to invoke the provisions of 35 U.S.C. §112, ¶6. Moreover,even if the provisions of 35 U.S.C. §112, ¶6 are invoked to define theclaimed features, it is intended that the features not be limited onlyto the specific structure, material, or acts that are described in theembodiments, but in addition, include any and all structures, materialsor acts that perform the claimed function as described in alternativeembodiments or forms, or that are well known present or later-developed,equivalent structures, material or acts for performing the claimedfunction.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present invention may be derived byreferring to the detailed description when considered in connection withthe following illustrative figures. In the figures, like-referencenumbers refer to like-elements or acts throughout the figures. Thepresently embodiments are illustrated in the accompanying drawings, inwhich:

FIG. 1 is a front view of a human eye.

FIG. 2 is a section view of a human eye from the side.

FIG. 3A depicts a top down view of an eye showing the regions of vision.

FIG. 3B shows an example approximation of the sizes of the regions ofvision.

FIG. 4 depicts overall system architecture.

FIG. 5 depicts architecture of the eye signal object.

FIG. 6 depicts abstracted hardware.

FIG. 7 depicts typical Bluetooth architecture.

FIG. 8 depicts hardware interface and hardware components.

FIG. 9 depicts imaging architecture.

FIG. 10 depicts biometric data generation.

FIG. 11 depicts a breakdown of a cognitive load manager.

FIG. 12 depicts system components of the cognitive load manager.

FIG. 13 depicts a HMD (head mounted device) connecting to a mobiledevice.

FIG. 14 depicts a HMD connecting to the cloud.

FIG. 15 depicts a HMD connecting to home and vehicle controls.

FIG. 16 depicts communication between a HMD and a NEST™ thermostatsystem.

FIG. 17 shows system architecture on a HMD communicating with the cloud.

FIG. 18 depicts a breakdown of the data manager.

FIG. 19 shows the system architecture of a HMD with the processingcapabilities moved to the cloud.

FIG. 20 shows further evolution of a HMD towards the Internet of Things.

FIG. 21 depicts the system architecture from a HMD moved to a remoteserver.

FIG. 22 depicts a HMD with all processing pulled off to the cloud.

FIG. 23 depicts a HMD and the remote server communicating.

FIG. 24 depicts a HMD communicating with home control systems.

FIG. 25 depicts a HMD communicating with social media.

FIG. 26 depicts a HMD communicating with home entertainment systems.

FIG. 27 depicts a HMD communicating with vehicle entertainment systems.

FIG. 28 depicts a HMD communicating with vehicle control systems.

FIG. 29 is a flow chart of steps taken to perform an online, securepurchase.

DETAILED DESCRIPTION

In the following description, and for the purposes of explanation,numerous specific details are set forth in order to provide a thoroughunderstanding of the various aspects of the embodiments. It will beunderstood, however, by those skilled in the relevant arts, that theapparatus, systems, and methods herein may be practiced without thesespecific details. It is to be understood that other embodiments may beutilized and structural and functional changes may be made withoutdeparting from the scope of the apparatus, systems, and methods herein.In other instances, known structures and devices are shown or discussedmore generally in order to avoid obscuring the embodiments. In manycases, a description of the operation is sufficient to enable one toimplement the various forms, particularly when the operation is to beimplemented in software. It should be noted that there are manydifferent and alternative configurations, devices, and technologies towhich the disclosed embodiments may be applied. The full scope of theembodiments is not limited to the examples that are described below.

The anatomy of the eye is well known in the art. For the purposes ofthis disclosure, relevant anatomy is depicted and described. FIGS. 1 and2 generally depict the anatomy of the human eye 100. FIG. 1 is a frontview of the eye 100 showing the pupil 145, iris 115, sclera 150, limbus245, pupil/iris boundary 250, upper eyelid 105, lower eyelid 110, andeyelashes 235. FIG. 2 is section view of the eye 100 showing the pupil145, iris 115, retina 52, sclera 150, fovea 160, lens 165, and cornea170. The pupil 145 is the approximately round dark portion at the centerof the eye that expands and contracts to regulate the light the retina52 receives. The iris 115 is the colored portion of the eye 100 thatsurrounds and controls the expansion and contraction of the pupil 145.The sclera 150 is the white region of the eye 100 that surrounds theiris 115. The sclera 150 contains blood vessels and other identifiablemarkers. The limbus 245 is the outer edge of the iris 115 next to thesclera 150. The pupil/iris boundary 250 is where the pupil 145 and theiris 115 meet. The eyelids 105, 110 and the eyelashes 235 surround andoccasionally partially cover or obscure portions of the eye 100 duringblinks, eye closures, or different angles of viewing. The retina 52 isthe sensory membrane that lines the eye 100 that receives images fromthe lens 165 and converts them into signals for the brain. The fovea 160is an indentation in the retina 52 that contains only cones (no rods)and provides particularly acute vision. The lens 165 is the nearlyspherical body of the eye 100 behind the cornea 170 that focuses lightonto the retina 52. The cornea 170 is the clear part of the eye coveringthe iris 115, pupil 145, and the lens 165.

FIGS. 3A and 3B depict the foveal, parafoveal, and peripheral ranges ofvision. The foveal region 190 is about two degrees outward from a user'sgaze point. An approximation of this region is a US penny held at anadult's arm length. The parafoveal range 195 is the viewable areaoutside the foveal region 190, generally from two to ten degrees from auser's gaze point. An approximation of the ten degree parafoveal visualfield is a circle with a four-inch diameter held at an adult's armlength. The peripheral region 197 is outside of the parafoveal region195 and is generally from ten to thirty degrees out.

FIG. 4 depicts an overall system architecture, including a processor1035 with non-volatile flash memory 1050, D-RAM memory 1040, a hardwareabstraction layer (HAL) 1030, and physical connections 1235 to externalhardware, an operating system (OS) 1025, and software and/or firmware1000 that handles the middleware services for the HMD that operates as aVisual Disambiguation Service (VDS) interface termed IRIS (Interface forReal-time Image Services) for the HMD and is operable as a softwarecontrol object. Above the middleware services is a software layer 1015containing software to facilitate the software integration of the IRISobject with a third party application; also above the middlewareservices is a set of software tools 1020 used for third party hardwareintegration and debug, including operations like single stepping andbreak-pointing through the Joint Test Action Group (JTAG)—supported bythe IEEE 1149.1 Standard Test Access Port and Boundary-ScanArchitecture. Above the software tools and integration layer is anapplication programming interface (API) 1010, followed by applications1005. Additionally, the system includes public 1635 and private keygenerators 1630 for added security.

FIG. 5 depicts the overall system architecture including software blocksidentified as a power manager 1140 and device manager 1120. Powermanagement schemes are derived from one or more open standards such asthe Advanced Configuration and Power Interface (ACPI).

The ACPI has three main components: the ACPI tables, the ACPI BIOS andthe ACPI registers. Unlike its predecessors, like the APM or PnP BIOS,the ACPI implements little of its functionality in the ACPI BIOS code,whose main role is to load the ACPI tables in system memory. Instead,most of the firmware ACPI functionality is provided in ACPI MachineLanguage (AML) bytecode stored in the ACPI tables. To make use of thesetables, the operating system must have an interpreter for the AMLbytecode. A reference AML interpreter implementation is provided by theACPI Component Architecture (ACPICA). At the BIOS development time, AMLcode is compiled from the ASL (ACPI Source Language) code. To date, themost recent release of the ACPI standard was in 2011.

For wearable computing, the systems of the future may implement withoutan operating system and robust support. A power management scheme andACPI elements discussed above will need to be pulled up to anapplication control level, giving the application and user dynamiccontrol of the power scheme. As discussed below, in highly abstractedembodiments, the ACPI might implement in a split or distributed fashion.This current standard does not fully anticipate the challenges ofwearable computing devices like an HMD disclosed in this specification;therefore additional considerations for operating HMD systems inmultiple modes is disclosed.

An HMD may include a low-power mode of operation that may be deployedduring times when no eyes are detected. This typically occurs when theuser removes the headwear or when the headwear has shifted out of placeon a user's head. This functionality could be implemented in silicon asa system on a chip (SOC).

At any time, the device can be re-mounted by the original user or wornby a new user. For purposes of device calibration (e.g., to account foranatomical variations among individuals) and/or user authentication, itis desirable for the device to be capable of determining the identity ofregistered users when re-mounted or re-deployed. This can includeloading a new set of configuration/calibration parameters anddifferentiating identities between the previous and new user; includinghalting, pausing and/or concealing the outputs of any ongoing programslaunched by the previous user.

Typically under the old standard, once the Operating System PowerManagement (OSPM) activates ACPI, it takes over exclusive control of allaspects of power management and device configuration. The OSPMimplementation also exposes an ACPI-compatible environment to hardwaredrivers 1197, which each in turn impacts the system, device, andprocessor states; and these are managed globally as Power States andthese include Global States, Device States, Processor States, andPerformance States.

Power consumption is an omnipresent concern, particularly if the deviceis not worn for an extended period. A commonly deployed solution to thisissue is an “off” switch that completely powers down an electronicdevice. However, the time and inconvenience of “powering up” a headsetdevice is restrictive particularly, for example, if the device has onlybeen removed from the head momentarily.

Low-power HMD and eye-signal control anticipates these issues by usingat least one technique comprising:

-   -   modifying the Processor States by reducing clock rates to        processor(s),    -   modifying Performance States by confining processing to a low        power processor or portion of a processor,    -   modifying Device States by imaging at a reduced frame rate,    -   modifying Global States by turning the camera off or into a        low-power mode between images,    -   reducing illumination,    -   collecting and/or processing images with reduced spatial        resolution,    -   limiting algorithms (particularly those associated with        searching for iris boundaries) to low-spatial resolution modes,    -   relaxing stringency measures during irisCode comparisons, and    -   fabricating specific dedicated hardware (a chip or SOC) that        operates in a low-power mode that does not “power-up” the full        device until a low-level authentication has occurred. Further,        such a chip or SOC could prohibit access to other embedded        functionality or connected or wirelessly connected devices until        authentication, possibly following power up and the        determination of a user's eye viewing a display or other target        object is performed.

This specific dedicated hardware can utilize modern methods of “hybrid”chip manufacturing that can segment a portion of circuitry to operate inan extremely low power mode. This hybrid circuitry effectively builds a“firewall,” preventing an unauthorized user from fully powering up orutilizing a device.

Another application of low-power HMD modes is when a low-battery stateis sensed. Instead of running a device until all global functions cease,a “graceful degradation” model is implemented as part of the new classof Power State for HMDs. “Graceful degradation” can include algorithmicapproaches by limiting the use of more power-hungry (i.e., generallymore sophisticated) image processing and other routines; as well as anynumber of the hybrid and hardware approaches to reduce power whilemaintaining at least partial functionality, discussed above. Low-powermodes for the processor and critical operations continue until thebattery finally runs out of power, the unit is plugged into a centralpower source, or the device is placed sufficiently close to an inductivecharging station.

Another power management concern for all forms of wearable computing isthat more sophisticated algorithmic eye tracking and user interfacetechniques can draw upon faster or parallel central processing units(CPUs), but generally these approaches require more power. Greater powerconsumption results in larger and/or heavier batteries, and/or shorterdevice use times between recharging or replacing batteries.

An alternative or adjunct to the deployment of more/faster CPUs is theuse of embedded or distributed processing approaches. These can beimplemented within a variety of hardware components includingfield-programmable gate arrays (FPGAs), application-specific integratedcircuits (ASICs), complex programmable logic devices (CPLDs) and hybriddevices that can include system-on-chip (SoC) configurations.

Embedded or distributed processing can facilitate existing, CPU-basedapproaches by off-loading computationally intensive routines. Hardwarededicated to performing these routines can be faster (often requiringonly one, or just a few, clock cycles) and utilize less power (often bygreater than an order of magnitude). Distributed processing can alsofacilitate new algorithmic approaches that are generally not feasible(within time and power-consumption constraints) using CPUs. Distributedprocessing is particularly valuable within algorithms that requirerepeated and/or simultaneous application of calculations to be performedon large data sets such as video images. These are further discussedbelow in the sub-section Distributed Processing.

Another embodiment utilizes low-power distributed processing to detectwhether the device has been removed from the head. In order to implementan “instant on” capability, the device must “sense” whether it ismounted on the wearer's head or has been removed. A method to performthis function (without adding additional hardware) is to determine if aneye can be viewed within eye-tracking camera images. As describedelsewhere, power consumption can be reduced when the device is not inuse (i.e. removed from the head) by a reduced frame rate, low resolutionimaging, lower CPU clock rate, etc.

For low-power eye-presence measurements, illumination can be eliminatedor reduced by reducing the power of illuminating LEDs, reducing thenumber of LEDs turned on and/or only turning on illuminator(s) whenactually sampling camera images (at reduced frame rates). A substantialreduction in power can also be attained by embedding relatively simpleeye geometry detection routines in distributed processing hardware. Anexample of this is one form of convolution filter to determine if animage (i.e. of an eye) is present is a focus filter. Such a filter wouldbe classified as a high-pass spatial filter that detects the presence ofhigh spatial contrast edges. The absence of such edges indicates thatthe device has been removed from the head when a defocused image isgenerally present (i.e. absent a high-contrast object located at theapproximately 25 mm focal distance of the camera). Another approach isto detect a dark (i.e. pupil) region adjacent to a white (i.e. sclera)region. When an in-focus eye is detected, the device “powers up”(recognizing that it was not completely powered off) for higherresolution eye tracking.

In another embodiment, the device may include a micro electro-mechanicalsystem (MEMS) such as an accelerometer or rate sensor for determiningmotion. When the device is not being worn it may operate at an ultra-lowpower mode in which it is not intermittently searching for the presenceof an eye. In the ultra-low power mode, the device may only search forthe presence of an eye when movement of the device is detected, forinstance when a user picks up the device. When movement is detected, thedevice may initiate a scan in search of an eye or eyes at predeterminedintervals (for instance every two seconds) or substantially continuouslyfor a period of time (for instance one minute) as set by userpreferences. If the device fails to detect an eye in the pre-set timeinterval it may resume ultra-low power mode or it may cycle through alow power mode prior to resuming ultra-low power mode. Should an eye oreyes be detected, the device will switch into full power mode or into apower settings scheme as set by the preferences of the detected user.The primary device owner (administrator) may set the overall systempower schemes that will govern the power mode settings for the devicewhen it is not in use. Additionally, the device owner may lock downchanges to the power schemes such that other users are unable to editthem.

FIG. 5 depicts a further breakdown of the IRIS Object 1000, includingeye tracking 1100, a module for tracking the eye gaze of a user; eyedata 1105 for user identification using biometric data of the user suchas facial, speech and iris identification; eye control 1100 for relatingthe user's eye gaze to a display, iUi™ interface (an interfacecomprising eye-signal controls) 1116. Eye signals 1115 gleaned from eyemovements are used to interact with a user interface iUi 1116 anddisplay screen(s) and images on the display.

Included in the IRIS object 1000 are a number of software modulesoperative as managers of certain functions. As an example, but notlimiting to:

-   -   Device Manager 1120 that allows a user to control hardware        attached to the system, whether it be the imager in the HMD, a        cell phone attached, or a vehicle Infotainment system.    -   Data Manager/Personal Data Management 1125 enables the secure        storage and access to user data such as e-mail, messages etc.        The Data Manager 1125 also may include one or more of password        management software, web browser favorites, and cryptographic        software. Advance data management can include, as an example,        setting and establishing a Virtual Private Network (VPN),        terminal services with an external processor, whether local or        accessible through the internet.    -   Communications Manager 1130 is designed to pass information from        one system to another and provides remote access to systems and        transmits files in a multitude of formats between computers. The        Communications Manager 1130 also may include link management and        data routing.    -   Security Manager 1135 refers to software steps or measures used        to protect the HMD and user from threats, viruses, worms,        malware, or remote hacker intrusions; to include        preventive-control techniques, which safeguard the system and        its data from being stolen or compromised.    -   Power Manager 1140 manages device power schemes to optimize and        maximize a user's experience and device battery life.    -   Display Manager 1145 has two basic functions: how the eye is        tracked to objects on the screen in the mode of the iUi 1116;        and what is displayed to the user in the HMD. The Display        Manager 1145 also has the ability to transfer all or portions of        the screen in the HMD to an external display, such as a computer        screen, the dashboard of a vehicle, or a home entertainment        monitor such as a TV.

With now a further discussion on the Security Manager 1135, a number ofadditional controls may be included, for example audio/video (A/V)control 1150; speech control 1155; or something more complex, e.g.,cognitive load control 1160 (FIGS. 11 and 12).

The HAL 1030 (FIG. 6) includes the “hosted” aspects of external hardwaresystems; this generally includes software specific to the IRIS platformdeveloped specifically for integration. The hardware anticipated, butnot limited to, is a Bluetooth interface 1170 (discussed separatelybelow as one embodiment), a TCP/IP interface 1175 to any form or anytype of TCP communications to include 802.2 (wired interface) 802.11,802.15 (WPAN—Wireless Personal Area Networks other than Bluetooth), and802.20 Wi-Max; this includes stacks that support the network andtransport software 1195 and the physical links to wired or wirelesssystems. In addition, there are considerations for other systems throughthe Hardware IF 1180, which interfaces with external software and/orhardware drivers through physical links 1235; these physical links canbe I²C, USB, serial, or proprietary.

In a discussion of one embodiment, as an example, the Bluetooth systemhas been selected in a non-limiting example of an embodiment because itis so pervasive in the rapidly growing market of mobile devices. As anexample, today almost all vehicles have what are called Infotainmentsystems; these are a combination of entertainment such as music andvideos as well as information, where the information could come fromwithin the vehicle as data from a sensor, control of a system like aheater or lights, or information available through the internet. Most ofthese systems use wired and wireless technologies to connect to thevehicle and/or the internet. Today, the wireless connections to thevehicle are generally Bluetooth established through a set of standardinterfaces; these are referred to as Profiles 1300 (FIG. 9) and arehosted in a processor above the Bluetooth radio, further shown in FIG. 5as Hands Free Profile (HFP) 1187; Advanced Audio Distribution Profile(A2DP) 1193, Audio Video Resource Control Profile (AVRCP) 1190, etc.

To date, it has not been anticipated that a HMD 600 would be used in avehicle to control vehicle operations 915, to include the Infotainmentsystem 910; therefore incorporation of one of the newest Profiles isinevitable, this Profile is called Human Interface Device (HID) Profile;profiles 1210, 1215, 1220, and 1225 exchange information from theapplications through the profiles across the hardware abstraction to theBluetooth module and Bluetooth radio. FIG. 6 depicts a Bluetooth systemsarchitecture, including connections to the profiles, network andtransport 1230, and the data link 1250 and modem 1255.

FIG. 7 depicts a breakdown of the Bluetooth architecture 1205 brokendown into their subcomponents. Underlying all of these protocols is akey piece of Bluetooth termed the Service Discovery Protocol (SDP) 1310,which includes what is called Secure Simple Pairing (SSP). SSP today isrequired by all Bluetooth standards above v2.1. Secure Simple Pairinguses a form of public key cryptography, which can help protect againstwhat is called “man in the middle,” or MI™ attacks. Generally, theBluetooth HID 1185 specification requires a security mode 4 for pairingand bonding two devices together citing that it should not be possibleto perform pairing or bonding to any Bluetooth HID Host or Devicewithout physical access to both the Bluetooth HID Host and Bluetooth HIDdevice. Bluetooth HID Hosts and Bluetooth HID devices that supportbonding use some form of non-volatile memory to store the 128-bit linkkeys and the corresponding BD_ADDRs, as well as the type of eachlink-key (authenticated, unauthenticated, or combination). In the caseof a HMD, complex access is limited as there is no mouse or keyboard ina conventional sense. However, there are other ways to establish asecure link that have not been anticipated by Bluetooth, even thoughBluetooth acknowledges the precepts of Public key cryptography.

In another example, a Bluetooth HID Host that accepts sensitiveinformation from Bluetooth HID devices may be implemented to only acceptsensitive information from reports that are contained in a top-levelapplication collection of “Generic Desktop Keyboard” or “Generic DesktopKeypad.” Furthermore, such a Bluetooth HID Host may require MITMprotection when pairing with any Bluetooth HID device with a BluetoothHID report descriptor that contains a top-level application collectionof “Generic Desktop Keyboard” or “Generic Desktop Keypad,” which in turncontains any of the following sets of usage codes and theirdescriptions:

-   -   IC—irisCode: the result of applying pattern-recognition        techniques to images of an eye to quantify the epigenetic        patterns within an iris into comparable bit-patterns for the        purpose of biometric identification.    -   EIC—Encrypted IC: an irisCode that has been encrypted so that it        cannot be reverse engineered to an original image of the iris or        any other iris-based, derived parameter.    -   TEIC—Target EIC: an identified EIC in which a match with an IC        computed from an image of an eye indicates association and thus,        a positive biometric identification.    -   CBID—Continuous Biometric Identification: the repeated process        of biometric identification that can be performed either on a        headset device or remotely by transmitting EICs, or images of        one or both eyes to a remote processor. CBID can occur at a        fixed rate (e.g. 30 times per second) or an asynchronous rate        (e.g. each time the device is moved or re-mounted).

The following table refers to the transmission and comparison of EICs;however, it is algorithmically possible to convert images of eyes intoICs and subsequently into EICs. Thus, CBID can equivalently involvecomparisons and/or the exchange of information involving images ofirises, ICs, EICs or other derived parameters. Similarly, databases usedfor biometric comparisons could equivalently (for the purposes ofidentification) contain ICs, EICs, images of eyes, images of faces(including eyes), images of irises, so-called “unfolded” (i.e. expressedin polar coordinates) iris images, or other derived parameters.Therefore, references to exchanges or comparisons of EICs also refer tothe exchange or comparison of any other derived data sets for thepurpose of biometric identification.

In Table 1 below, where a set of programmable attributes can be assignedto a new Bluetooth profile, these attributes can be used:

TABLE 1 Example Headset Configurations for a Bluetooth SDP TransactionAttribute Attribute ID Value Description Example ExtendedProto- 0x360x0C Data element Following the colDescriptor sequence, BT HID 1.1 List12 octets 0x020F (Next BT HID Revision) stand-alone, 0x36 0x01 Boolean8,recognizes the 1 of 1 single TEIC sole owner/user stored within of adevice headset with that is no off- inoperative headset otherwisecommunication required for CBID

Other examples of programmable attributes include but are not limitedto:

-   -   recognize an individual member of a family, all of whom are        permitted to use a device (e.g. dynamically loads calibration        factors associated with each user)    -   pay-per-view rental of a public HMD    -   enabling multiple, general-use headsets available to employees        within a business    -   online purchase from an online “store” in which a user has been        registered with no restrictions on the device used to make a        purchase    -   online purchase from an online “store” in which both a user and        specific headset have been registered    -   determine if user is on a “no-fly” list    -   confidential list of traditional passwords    -   taking an examination within a massively online course    -   medical records made available to primary care doctor,        specialist, and patient    -   structured advertising based on the demographic of an identified        viewer    -   a device license agreement sent to a user    -   confirmation that all components of a legal document have been        viewed    -   confirmation that a notice of changes in terms and conditions        has been sent to a user    -   confirmation of informed consent related to legal documents    -   pre-flight inspection by a pilot    -   identification of a vehicle driver and possibly identifying        pre-accident driver distractions    -   retrieve e-mail based on user identification where another user        would be offered separate e-mail access    -   outgoing text and e-mail can (optionally) be tagged to indicate        the CBID user is the author    -   electronically “sign” legal documents    -   administration of an examination that must take place in the        presence of both student and an instructor    -   exchange of personal information between/among people who have        just met    -   purchases made in a bricks-and-mortar store that requires no        check stand    -   remote control of door opening for authorized personnel only    -   gaining or restricting access to a building based on user        identification    -   tracking an individual under house arrest    -   tracking an individual restricted from entering a casino or        interacting with another individual    -   ensure legal purchase of alcohol or other age-sensitive        materials to an individual    -   automatic 911 call with user identification (that can be linked        to medical history), “vitals” and geographic location    -   based on CBID, interact with an automated teller machine    -   gaining access to highly secure military sites    -   proving the historical activities of an individual under        investigation    -   restricting access to the audio/video of a private conversation    -   restricting access to the audio/video of a conference to        participants    -   restricting access of a data set of a private conversation to        CBID participants    -   when/where is the last time I saw my car keys?    -   list hockey (versus basketball or some other sport) scores first    -   control household thermostat by an adult (not a child)    -   remotely turn on household entry lights

In a further discussion of the image process and use of the imager orvideo, in the embodiment described above, a system is anticipated wherethe HMD is implemented with a single conventional 2D imager systemoriented either toward the eye and face and outward facing to the scene.However, in an alternate embodiment, consideration is anticipated forthe HMD to be implemented with multiple imagers oriented toward the faceand eyes as well as the scene, where the multiple imagers generate astereoscopic 3D video image. In addition to stereoscopic 3D images,consideration for other forms of 3D image generation has beenanticipated by the applicant. Today, non-contact three-dimensionalcameras, or digitizers, generally fall into four categories:stereoscopic digitizers (as mentioned above), silhouette digitizers,timing digitizers, and projected pattern digitizers. The underlying 3Dsurface imaging technologies can further be summarized in terms of fourbroad categories: Spatial Phase Imaging (SPI), Triangulation, Time ofFlight (TOF) and Coherent approaches.

Spatial Phase Imaging generally relies on the polarization state oflight as it emanates from surfaces to capture information about theshape of objects. Triangulation employs the location of two or moredisplaced features, detectors, and/or illuminants to compute objectgeometry. Two important triangulation subcategories are stereocorrespondence (STC) and stereoscopy (STO). Stereo correspondencecameras determine the location of features in a scene by identifyingcorresponding features in two or more offset intensity images using 3Dgeometry to compute feature locations. Stereoscopic cameras rely onhuman biological systems (eyes, brain) to create a notion of a 3D scenefrom two images taken from different vantage points and projected intothe eyes of a viewer. Finally, coherent methods rely on a high degree ofspatial and/or temporal coherence in the electromagnetic energyilluminating and/or emanating from the surfaces in order to determine 3Dsurface geometry.

FIGS. 8 and 9 depict imager object code 1415 for either a 2D or 3Dimplementation. Regardless of the technology employed, any systemimplemented must consider two key factors: Human-fidelic visualization(completely realistic display) and visual intelligence (automatedvision). Human-fidelic visualization can create a visual notion of ascene in the mind of a human that is as realistic or almost as realisticas viewing the scene directly; the visualization system ishuman-fidelic. An imaging system has to be 3D to be human-fidelic, sincehuman sight is 3D. The second being visual intelligence, which meanssensing and analyzing light to understand the state of the physicalworld. Automatic recognition of human emotions, gestures, and activitiesrepresent examples of visual intelligence. 2D video cameras struggle toprovide a high level of visual intelligence because they throw awaydepth information when a video is captured. As a consequence ofneglecting depth, 2D images of 3D scenes are inferior to 3D images. 3Dimages have better contrast (the ability to distinguish betweendifferent objects). Real video of real scenes typically contains dozensof instances where contrast and depth ambiguity make it difficult forautomated systems to understand the state of the scene.

3D video cameras do everything that 2D cameras do, but add the benefitsjust discussed. It is inevitable that single lens native 3D video willeventually replace 2D video offered today by offering two interestingbenefits: human-fidelic visualization and improved visual intelligence.It is reasonable to assume that global production of most cameras willshift to 3D as they become cost effective, simple to operate, compactand produce visual fidelity. With this in mind, the technology emergingtoday as the most likely to reach mass markets in terms of cost,complexity, and fidelity is Spatial Phase Imaging within the broad 3Dimaging categories discussed. This technology relies on commerciallyavailable imagers implementing a micro-polarizing lens over foursub-pixels resulting in an ability to rapidly determine small changes inreflected light, computing a vector as a direction cosine for each pixeland generating a three dimensional value in terms of X, Y and Z-depth;truly a single lens native 3D video.

In another embodiment, the accuracy of both CBID and eye-signal controlprocesses can be improved via the use of more than a single camera toview an eye. Images substantially simultaneously or sequentiallyacquired from multiple cameras can be used to 1) create on-axis (i.e.perpendicular to the surface) views of different regions of the eye, 2)view surfaces with specular reflections (particularly glints) located atdifferent positions within images of the eye, 3) allow for viewing offine structures while maintaining the ability to view over a widespatial range, 4) increase eye tracking accuracy by making multiplemeasurements based on multiple views of glints and eye structures, and5) view “around” obscuring objects such as eye lids and lashes.

Another area where distributed/embedded processing is particularlyvaluable is in the “off-loading” of operations that are computationallyintensive for a CPU. Examples of such a “hybrid” approach (i.e. mixingCPU and embedded processing) within eye tracking and iris identificationalgorithms include subroutine that perform Fast Fourier Transform (FFT),random sample consensus (RANSAC), so-called StarBurst featureextraction, and trigonometric functions.

Included within the system architecture are methods for managingcognitive load for safety, optimized performance, and general well-beingfor individuals and groups. Generally, the concept of cognitive loadextends from tactical fighter programs and activities that generallyrelate to situation awareness. These in turn drive cognitive loadcontrol 1160 in programs like cockpit workload management; cognitiveload control 1160 generally deals with the human mind interacting withsome external stimulus.

The definition of cognitive load is slightly different in differentfields; for example, in an academic sense cognitive load refers to thetotal amount of mental activity imposed on working memory at anyinstance in time; while in the ergonomics literature it is described asthe portion of operator information processing capacity, or resourcesthat are required to meet cognitive task demands. Each field providesdifferent methods to measure cognitive load.

Cognitive load is considered herein as the mental effort or demandrequired for a particular user to comprehend or learn some material orcomplete some task. Cognitive load is relative to both the user (i.e.,their ability to process novel information) and the task being completed(i.e., complexity), at any single point in time. It is attributable tothe limited capacity of a person's working memory and their ability toprocess novel information.

Conventional methods for measuring cognitive load, include:

-   -   1. subjective measures, such as self-rating scales;    -   2. physiological techniques, such as pupil dilatation, heart        rate and galvanic skin responses;    -   3. task or performance based measures, such as critical error        rates and task completion times; and    -   4. behavioral measures, such as speech pathology (e.g.,        impairment, self-talk, etc.)

There are a number of problems with these methods for measuringcognitive load, including:

-   -   1. some of the methods are intrusive and disrupt the normal flow        of performing the task;    -   2. some of the methods are physically uncomfortable for the        user;    -   3. some methods cannot be conducted in real-time as they are too        intensive;    -   4. the data quality is potentially unreliable outside laboratory        conditions; and    -   5. the data quality can be affected by outside factors, such as        a user's stress level.

FIG. 12 depicts system components for a cognitive load manager 1160 thataddresses many of these issues. In one embodiment, mobile, wearable,implanted, consumed, and other physiologically integrated computersemploy increasingly sophisticated and varied sensors, data inputmethods, data access methods, and processing capabilities that capture,access, and interpret more and more data that can be used as sensoryinput to the brain and impact cognitive activity. The data comprisesphysiological data 1815 and environmental data 1810. The data are usedto better establish a user's preferences for the integration,management, and delivery of information to the head mounted unit.

FIGS. 13-15 depict three different system architectures for connectingthe HMD to another device or to the Internet. FIG. 13 depicts the HMD600 connecting through a local link, such as Bluetooth, to a mobiledevice 710 carried by the user; the mobile device 710 is connected vialink 155 to a packet switched network typically provided by a wirelesscarrier through 700 or what today is generally referred to as a packetnetwork also known as the world wide web; with subsequent connection toeither a web-based service, a database, or external application 160.

FIG. 14 depicts the HMD 600 including a wireless transmitter 750 that iseither embedded or attached to the HMD for connection directly to theinternet 700 and a service provider 160.

FIG. 15 depicts the HMD 600 including a wireless transceiver 750connected via a link 725 directly to the Internet, where the local linkis generally a packet link, but could be other proprietary wirelessprotocols. In this configuration, the HMD is independent from othersmart devices; essentially the HMD is connected directly to the Internetall of the time. Today, if a user wants to connect a smart device, ornow a HMD, to another system for the control and operation of thatsystem, the user would simply implement a local connection through aBluetooth profile. In the case of home audio the user would need to useAudio Video Transport Profile, Audio Video Resource Control profile, orAdvanced Audio Distribution Profile. If a user wanted to connect to avehicle, he or she would need to implement the Hands Free Profile.Simpler and less complex systems are needed along with methods toconnect to these systems, especially if the user is beyond the range ofa local connection to the system they want to control.

To solve this new challenge, FIG. 16 depicts another embodiment where anHMD is implemented in an “abstracted” real-time server-browser cloudbased architecture; known today as the “Internet of Things” or IoT. Thekey to any abstracted layer is the ability to abstract away from somedevice or software operational or strategic complexity; these couldinclude proprietary aspects, including trade secrets and intellectualproperty. The abstraction can support extended or new business models toa technology supplier. A good example of this architecture is the NEST™Labs business model. This model could be loosely referred to as a“razor/razor blade” model; in this case the NEST™ thermostat is therazor, the NEST™ Services are the razor blades or simply stated thebusiness model includes the sale of the thermostat and a monthlyrecurring service. In addition to the sale of hardware and services,this business model supports data harvesting of a user in his home. Inthis system, the thermostat serves data off to a centralized server forthe purposes of “learning.”

Even though NEST™ products can be accessed via the Internet, they cannotbe directly connected to by a smart device for the control and operationof a home heating system. FIG. 16 depicts an HMD 600 connected via apacket network 155 to the Internet 700. In order for the user to accesshis or her home thermostat, the user needs to access their page on theNEST™ Services server 965. However, the traditional role of web serverand browser has been expanded under the new HTML 5 standard. There hasbeen what looks like a role reversal of the server and browser, onewhere the web server is now the smart thermostat; this server is simplyserving small amounts of data to a fixed URL in the cloud running abrowser. This browser in the cloud can be accessed by a user using asmart device or computer from virtually anywhere to read or interactwith their thermostat. Using the web server in this role is now a keyand underlying concept of the IoT, one where complexity and cost aregreatly reduced.

Now re-thinking FIG. 15, in view of the IoT; access to homeentertainment, home security systems, or for that matter any homeappliance (washers, dryers, refrigerators, etc.) for their monitoring,control, and operation will implement differently. Further, consideringthe IoT architecture, the head-mounted device 600 could be connected toany consumer, industrial, or commercial device located anywhere in theworld on the cloud, a user could control that device via eye interactionwith the included display via eye signals defined as a standardizedcommand set mapping the eye signals to communication, diagnostics,control, and interaction with the device(s).

This new model abstracts away complexity and cost, as an example, amodel where the HMD may not require Bluetooth or, for that matter, adistributed intelligence. It is inevitable two things will happen in thenear future: first wireless bandwidth will continue to growexponentially with gigabit service on the horizon; and second, the IoTarchitecture will deploy as it continues to deploy today; very rapidly.What are needed are methods and systems disclosed on how a standalonehead mounted system will strategically evolve within a rapidly evolvingecosystem. FIGS. 17-23 depict an abstraction transition model from asmart head mounted system, to a much simpler model as depicted in FIG.23.

Starting with the end of the transition first, FIG. 23 depicts a cloudbased implementation within an IoT architecture of an HMD 600 connectedby a very high speed packet based link, a wireless link that would rivalor potentially outperform the typical communication bus in a localprocessor. These processor busses operate as subsystems of the processorto facilitate transfer of data between computer components or betweencomputers. Typical bus types include front-side bus (FSB), which carriesdata between the CPU and memory controller hub; direct media interface(DMI), which is a point-to-point interconnection between an integratedmemory controller and an I/O controller hub in the processor; and QuickPath Interconnect (QPI), which is a point-to-point interconnect betweenthe CPU and the integrated memory controller. Other high speed busseshave been used in the embedded computing industry to include SPI forinter-processor communication. What is not currently anticipated is thatunder cloud based architectures and distributed computing, much of theintelligence will reside outside of the connected devices. HTML 5 andJSON are good examples of markup languages optimized for distributedcomputing. To include audio, video, and scalable vector graphics,operating systems will evolve and operate to meet these new distributedarchitectures likely using much simpler “publish subscribe” access.

With the above in view, and with respect now to the HMD operating on thecloud, the HMD is connected to a centralized server-browser 800 thatoperates the Visual Disambiguation Service (VDS) interface termed IRIS(Interface for Real-time Image Services); think of this operating muchlike SIRI (Speech Interpretation and Recognition Interface) does foraudio. The IRIS service is for the complex disambiguation of eyemovement for the real-time interpretation, determination, and predictionof a user's intent. IRIS, like SIRI, operates in the cloud. 1126 and1127 represent the IRIS abstraction layer discussed above. The HMD nowoperates with a minimum amount of software, a processor richer infeatures and configured with a limited or possibly no operating systemusing a publish/subscribe messaging scheme.

At the beginning of the transition, the embedded IRIS (e-IRIS) 1111includes a number of tools or utilities operating in the FOG as acombined real-time service. These include a data manager 1125, devicemanager 1120, communication manager 1130, power manager 1140, andsecurity Manager 1135 a. In the e-IRIS abstraction 1127, there arecounterpart managers, with a slight exception in the security manager1135 b; this will be discussed below in more detail.

FIG. 23 also depicts the eye management tools centralized in acloud-based version in support of a user. These include an eye tracker1100, eye data 1105 in support of security, eye control 1110, eyesignals 1115, and iUi 1116 for an eye user interface. In addition tothese elements, other real-time services are available and associated toIRIS including an Audio-Video manager 1150, speech manager 1155,cognitive load manager 1160 and a context manager 1165. The combinationof these services and architecture constitutes IRIS.

Back now to FIG. 17 and an initial embodiment, the HMD 600 is wirelesslyconnected to a smart device (such as a smart phone, a tablet, home oroffice PC) or simply to the Internet through an 802.11 link. All of theservices operate in the HMD 600 processor or are stored in a memoryassociated with the HMD 600. This embodiment would operate as astand-alone computer, with an operating system, and micro-processor(s)and/or other logic elements. In a first transition step of the firstembodiment, some of the non-real-time applications are off loaded toapplications run on the local smart phone 710, local PC, or other smartdevices. However, this first transition embodiment would still be highlydependent on the locally available resources in the HMD 600 to operateas intended.

FIG. 18 depicts a second transition step wherein the data manager 1125takes on a new role. In addition to managing data on and off the HMD600, the data manager is configured to manage some of the data either onor off board the HMD 600 using a markup language, such as or JSON (JavaScript Object Notation), HTML 4.01, or 5.0. The object of thistransition step is to implement a web server-browser relationship in theHMD 600. In this case, some of the data acquired by the imagers, audioinput, or any other sensors available to the HMD 600 are served to thecloud and directed by a fixed URL to a cloud based IRIS, where a user'sbrowser page resides and his/her data are aggregated. This secondtransition supports non real-time data applications, as an example theHMD 600 is used for the transmission of data that have been collectedand stored by a user. As an example, the user may capture a photograph,an audio clip, a video clip, or other user physiological data related tothe eye or a user's health; these data are then transferred to IRIS forstorage, aggregation, or possible subsequent dissemination (discussed inmore detail below).

FIGS. 19, 20, and 21 depict a third step in the transition, where thewireless bandwidth is now near real-time. A web server and browserrelationship exists operationally in parallel with a now more maturee-IRIS 1111 in the HMD 600 and IRIS 800 in the cloud. They operate andinteract with each other in near real-time across the abstraction layer1126 and 1127. This new configuration now allows an evolution of thesecurity manager with respect to security and implementation of theprivate key-public key. The security manager 1135 resident in the HMD600 takes on the role of generating a private key and public key basedon certain bio-metrics as disclosed in described in Systems and Methodsfor Discerning Eye Signals and Continuous Biometric Identification,filed May 8, 2015. Data collected from the face, eye, or voiceconstitute unique biometric data of the user or user groups if desired.These data collected can be used to generate a unique private key in asystem of public key and private key cryptography.

As background, cryptographic systems have been widely used forinformation protection, authentication, and access control for manyyears. These cryptosystems are generally categorized as symmetric keycryptosystems and public key cryptosystems. Symmetric key cryptosystemsuse the same key for encrypting and decrypting secret information;however using the same key can be problematic: 1) if the key iscompromised, security cannot be assured; and 2) if there are multipleusers, multiple keys are needed, which may increase system costs anddata security. Public key cryptosystems can overcome these limitationsby using a pair of cryptographic keys (i.e., a private key and a publickey). The private key used for decryption is kept secret, whereas thepublic key used for encryption may be distributed to multiple users.Therefore, secrecy of the private key is a major challenge when it comesto achieving high levels of security in practical crypto systems.

As one example, the irisCode of the user possibly combined with otherbiometric data are used to establish a unique key that subsequentlygenerates the private key-public key. The public key generated from theuser's unique biometric aspects is sent to IRIS 800 for storage in thesecurity manager portion of the user's browser, FIG. 22 1135 b. Theprivate key is never stored, but is generated in the HMD 600 every timea user instantiates a session. When the user dons the HMD 600, theprivate key is generated, FIG. 21 1129, and authenticated in IRIS 800.This ensures levels of non-repudiation and security currently notavailable in web applications, especially in e-commerce.

FIG. 23 depicts the final step in the transition to a real-time HMD 600.The Internet is now prolific and operates at speeds in excess ofprocessor buses. IRIS 800 is cloud-based and real-time for all intentsand purposes. Data are collected and aggregated in IRIS 800. IRIS 800 isnow implementing advanced algorithms based on learning about thephysiology of the human eye, as well as the user generally;disambiguation in IRIS 800 is enhanced to the point. IRIS 800 can nowpredict what and where a user wants to see or do. The user's HMD 600 iscommodity, low cost, low power and immediately replaceable.

The final step abstracts all of the intelligence for the device to thecloud 700. CBID, now cloud 700 based, is substantially continuous andreal-time. Since the generation of the private key is unique to a user,this allows any user to pick up any HMD 600 and use it at any time;simply slip it on and they are looking at their browser page where allof their personal information now resides. If their HMD 600 is stolen,the information is secure. If a user loses their HMD 600, no worry,simply borrow one, or buy a new one. The CBID and cloud 700 aspects ofIRIS 800, abstract the device at a new level, it abstracts the user likeHMI and displays do today.

As discussed above in the NEST™ home thermostat model, the thermostat isonly accessible through the NEST™ Services portal and page. In thisimplementation, the HMD 600 is securely connected to IRIS 800 and auser's page. If the user wants to access their thermostat, IRIS connectsthem directly and securely to the NEST™ Services portal 965. This modelwill extend to XFINITY, if a user wanted access to his/her account toset a recording, or have access to an XFINITY service, IRIS will connectthem directly the XFINITY portal 970. Further, if the user wants accessto their COZYHOME application, again, the link is securely made to theappropriate server in this case 975.

As discussed above, IRIS 800 may be linked to a user's social mediaaccount, giving the user a real-time access. FIG. 25 depicts how IRIS800 would securely connect a user securely to their Google+ account tosee postings or to post in near real-time information they want toshare. Social Media 920 comprises social media services available to auser.

Shifting now to real-time cloud based IRIS 800 and its extendedcapabilities, eye signals will be substantially continually aggregatedand analyzed for its users. This makes IRIS 800 a unique service anddevelopment platform for applications and services associated withcontextualized eye data (CED). IRIS 800 includes a context manager 1165in both e-IRIS 1111 in FIG. 17, as well as IRIS 800 FIG. 23 and its roleto generated Contextualized Eye Data (CED). CED begins with eye dataextracted from episodic and/or substantially continuous monitoring ofone or both eyes. These eye data include eye movements such as:saccades, fixations, dwells, pursuits, drift, tremors, andmicro-saccades. Eye data also include blinks and winks, squints, pupildilation, blood vessel patterns, iris and pupil size, feature locations,internal eye-structure size, shape, and location. A key aspect for CEDis to use this data to detect behavior changes over time.

CED is the correlation of eye-data with other classes of data over timeto extract relationships for meaningful prediction, measurement,analysis, interpretation, and impact on the user. As an example, threeclasses of data IRIS 800 will have aggregated are raw data, semanticdata, and evoked data.

Raw data comprises data captured by any sensors, whether in the HMD 600or present on or in a person. Today, there are many new wearable sensorsuses in sports or health where these new systems all have wirelesscapability. IRIS 800 can take this raw data from an individual andcorrelate it with eye data. Examples include, but are not limited to,sensors that capture: movement, GSR (galvanic skin response),temperature, heart rate and heart rate variability (HRV), EOG(Electro-oculogram), EEG (Electro-encephelogram, EKG(electro-cardiogram), temperature, facial muscle movement and skinmovement, internal organ or biological systems status and performance,scent, audio, scene and images for a range of electromagnetic radiation(visible light, IR, UV, and other electromagnetic frequencies), location(GPS and other beacon sensors), time monitoring/tracking, and more.

Semantic data comprises the interpretation or meaning of “what, when,where, and how” a user is “doing” something, as well as with whom theuser is doing something. “Doing” can be working, playing, eating,exercising, reading, and myriad other activities. These data areconstructed by interpreting sensor data in the context of a user'sactivities.

Evoked data are extracted from conscious or subconscious individualresponse to visual, tactile, olfactory, taste, audio, brain, or othersensory, organ, or biological responses to intentional stimuli.

To date, the capture of data associated with “eye-tracking” has beenprimarily enabled with expensive, stationary, “remote” eye-trackingdevices, situated in front of displays oriented towards users eyes, forlimited durations (measured in minutes) for specific tests; orexpensive, dedicated purpose, wearable eye-tracking devices, sometimespackaged as glasses, placed on users for limited durations, in limitedcontrived environments, for specific tests.

Eye-tracking data have primarily been captured indoors due to thetechnology's inability to function well in high-infrared (outdoor)environments without substantial filtering or shielding of ambient IRlight, further reducing the practicality, breadth, and quantity ofeye-data capture. As such, high quality, environmentally diverse,high-volume data across diverse “natural” use cases have been limiteddue to the expense, limited portability, constrained form-factor,high-power requirements, high-computing requirements, limitedenvironmental robustness, and dedicated “data capture” utility ofeye-tracking technology and devices. While early research on the datacaptured has shown promise for extraordinary insights into human health,cognition, and behavior, the general capture of such data has beenhighly constrained to specific tests and environments for shortdurations.

The first generation of IRIS integrated HMDs may be worn by millions ofpeople in a broad range of life activities. In the disclosed transitionplan, these data may be collected by IRIS first as historical data, thenin both near real-time, and ultimately in real-time. Should thistransition occur, it could increase by orders of magnitude the quantity,quality, and contextualization of eye data that is captured. IRIS couldthen have the ability to correlate data with a broad range of otherpersonal and aggregated data such as individual and group healthcognition and behavior. IRIS may then use the aggregated data to provideinsights into eye data correlated with personal health, cognition, andbehavior as a starting point regarding self-quantification,self-improvement, and self-actualization.

IRIS will support applications for extracting patterns from largedatasets that will expose and predict future behavior such as that ofour likelihood to adopt a new habit, our interest in acquiring aproduct, or our likelihood in voting for a new politician. As anexample, below lists the types of measurements and predictions that willbe afforded by IRIS' contextualized eye-data; these include but are notlimited to:

-   -   MEASUREMENTS        -   Measuring drowsiness and fatigue        -   Measuring medical conditions and trends        -   Measuring reaction to a drugs, food, or other comestibles        -   Measuring short, medium, and long term health trends        -   Measuring reading speed, focus, interest, fluency,            vocabulary, areas of confusion        -   Measuring knowledge, understanding, and skills        -   Measuring emotional state and reactions to stimuli        -   Measuring interest and emotional reaction to people, places,            and things        -   Measuring recognition and familiarity with people, places,            and things        -   Measuring focus and cognitive load        -   Measuring improvement in performance of specific and general            tasks        -   Measuring effectiveness and satisfaction with IRIS    -   PREDICTION        -   Predicting the onset of a medical condition or disease        -   Predicting the incidence of a specific health event such as            a seizure or panic attack        -   Predicting weight gain or loss        -   Predicting the general improvement of health        -   Predicting the likelihood of adopting a new behavior or a            bad habit        -   Predicting the likelihood of succeeding at a task or            endeavor        -   Predicting an automobile accident        -   Predicting the rate of improvement of an athletic skill        -   Predicting the market success of a new product        -   Predicting the rise or fall of a specific stock or the stock            market        -   Predicting a political outcome, political stability, and            political unrest        -   Impacting learning, work, play, understanding,            socialization, creativity, energy, focus, attitude,            motivation, and all things that make us human today, and            that will drive the enhancement and evolution of humanity            and our species.

The IRIS application and tools positively impact the user of the HMD bycontextualizing the eye data that are aggregated. IRIS technology willadvance the user's performance in many dimensions and will enhance theirhuman-to-human interactions as well as their human-machine interactions.

The key common aspect to any of these is IRIS's role as a real-timesecure abstraction. FIGS. 26-28 depict other portals for secure accessto a user's information where again, the common element is IRIS 800.Further, the private key stored in IRIS can be related to a password forthe user that greatly simplifies the user's interaction on the web, toinclude secure transactions.

In accordance with other embodiments; systems and methods are providedto enhance security and convenience during online shopping. FIG. 29depicts a user operating a setup process that needs to occur only oncewhere the user needs to link their public key with account information.For increased security, a bank or other financial institution that isresponsible for the account might verify other forms of target (i.e.,intended) user identity and offer the linkage process as a service. Oncelinked, online purchase selections and transactions can be performed bya user with their HMD in a seemingly instantaneous fashion.

In another embodiment of secure shopping, real time knowledge of adevice-wearer's identity allows financial particulars to be exchangedelectronically with each item as selected and purchased. This eliminatesthe need to repeatedly enter passwords, security questions or accountinformation for each transaction or group of transactions. As aconsequence, such an instantaneous purchasing system eliminatesprocesses involved with a so-called online shopping “carts” since thereis no longer a need to cluster items for the purpose of entering accountinformation. Solely for customer convenience, groups of items purchasedduring an online shopping session can be treated as a cluster orsummarized for the purchaser.

In accordance with another embodiment, systems and methods are providedto enhance security and streamline shopping at so-called “bricks andmortar” retail outlets. In this case, a camera mounted on the headweardevice that views the environment of the device wearer can be used toidentify objects that may be of interest for purchase. Identificationcan be based on bar codes or quick-response (i.e. Q-R) codes that arecommonly attached to purchasable items. Such object identification usesimage processing methods that are well known in the art.

Information about the item including a proposed purchase price can begenerated by a processing unit associated with the retail outlet. Thisinformation can then be displayed on nearby monitors or on ahead-mounted display associated with the device wearer. If the customerwishes to purchase a given item, a CBID-based transaction can beinitiated by the customer. Such transactions can occur repeatedlythroughout a store. A match between transported items and thetransaction record would then allow items to be verifiably removed fromthe store by the customer. CBID-based retail purchases eliminate theneed for check stands or tills. In many situations, the automated, realtime display of information during the purchasing process also reducesthe need for store clerks to assist potential customers.

These devices are also integrating increasingly sophisticated and varieddata output methods that stimulate visual, auditory, tactile, olfactory,gustatory (sense of taste), equilibrioception (sense of balance), directneurological, indirect (wireless) neurological (neural and synapticbrainwave stimulation), chemical, biological activity, and multi-modalinput sensation.

The increased stimulation of the body and associated enhanced deliveryof information to the brain can affect brain activity in subtle andprofound ways. Cognitive stimulation resulting from more, varied, andfaster delivery of multiple forms of input to the brain can positivelyimpact human performance. However, cognitive overload or inappropriatestimulation, can negatively impact performance, damage health, createsafety hazards, and even kill.

As mobile, wearable, implanted, consumed, and other physiologicallyintegrated computers proliferate, a solution is needed to managestimulation and flow of data to the body and brain. Individuals arealready applying various forms of cognitive management intechnologically stimulated situations. Some methods are purely manual,while methods for intelligent, software-based management are beginningto emerge. For example, reducing audio stimulation during periods ofincreased, high-impact cognitive activity is commonplace. Consider adriver of an automobile turning down the radio when driving stressincreases in challenging traffic or when a driver is lost and is tryingto navigate. The attention directed to listening, consciously orsubconsciously, provided by the audio stimulus of the radio, reducesinput to other areas of the brain, such as visual processing.Simultaneous multi-modalities, such as talking on a cell phone, impactthe visual task of driving.

Reducing physical exertion during periods of higher cognitive load isanother form of self-management that is commonplace. Research on“walking while talking” (WWT) shows a correlation between gait pace andrate as walkers talk. In general, walkers that become engaged inconversations requiring higher cognition typically slow their walkingpace.

A recent form of cognitive load management associated with electronicstimulation includes applications that temporarily disable email, text,and other online forms of interruption. These applications are verysimple in form, however.

This approach allows user's customization and prioritization to improveover time as historical context, performance, biometric, and other dataare accumulated and analyzed forming generally a user profile ofactivities and preferences. These also provide a variety of methods andtechniques for dynamically managing stimuli (deferral, termination,sequencing, reprioritization, pacing, and more), support stimuliaggregation and management across multiple individuals forrisk-controlled or performance-enhanced group activity.

Another embodiment is context-aware computing as 1165. In a mobilecomputing paradigm it will be advantageous for applications to discoverand take advantage of contextual information such as user location, timeof day, neighboring users and devices, user activity to specificallysupport collecting and disseminating context and applications that adaptto changing context.

For the sake of convenience, the operations are described as variousinterconnected functional blocks or distinct software modules. This isnot necessary, however, and there may be cases where these functionalblocks or modules are equivalently aggregated into a single logicdevice, program or operation with unclear boundaries. In any event, thefunctional blocks and software modules or described features can beimplemented by themselves, or in combination with other operations ineither hardware or software.

Having described and illustrated the principles of the present inventionin embodiments thereof, it should be apparent that the present inventionmay be modified in arrangement and detail without departing from suchprinciples. Claim is made to all modifications and variation comingwithin the scope of the following claims.

We claim:
 1. A system for a dynamically evolving cognitive architecturebased on a natural intent eye movement interpreter for real-time imageservices, comprising: one or more processors; and at least onestructured illumination source, operatively coupled to at least one ofthe processors and configured to be disposed towards at least a firsteye of a user; and a first sensor operatively coupled to the one or moreprocessors, wherein the sensor is configured to be disposed towards theat least first eye of the user; and non-transitory computer readablememory having a plurality of executable instructions stored thereon,wherein the instructions, when executed by the one or more processors,cause the one or more processors to process eye signal operationscomprising: configuring the one or more processors to operate in a firstprocessing mode; and receiving from the first sensor a first eyemeasurement data set representative of a first state of the first eye;and processing the received first eye measurement data set, wherein theprocessing calculates a predicted second state of the first eye, whereinthe second state is calculated to occur at one or more of a future date,future time, future interval, future position, future sequence, andfuture pattern; and responsive to calculating the predicted second stateof the first eye, sending a first executable instruction to the one ormore processors; and, receiving from the first sensor a second eyemeasurement data set, wherein processing the second eye measurement dataset calculates a second state of the first eye; and responsive tocalculating the second state of the first eye, correlating the predictedsecond state of the first eye with the second state of the first eye;and responsive to correlating the predicted state of the first eye andthe second state of the first eye, wherein the correlation resultdetermines the first and second eye states are congruent, sending asecond executable instruction to the one or more processors.
 2. Thesystem of claim 1, wherein the eye measurement data comprises at leastone of: eye lid data, eye lash data, pupil data, cornea data, retinadata, iris data, eye movement data, eye shape data, point of regarddata, dwell data, fixation data, saccade data, and illumination data. 3.The system of claim 1, wherein the eye signal operations performed bythe processor are at least one of continuously, simultaneously, andperiodically.
 4. The system of claim 1, wherein the first state of thefirst eye is a first eye gaze position.
 5. The system of claim 1,wherein the eye measurement data includes the detection of at least oneglint from the surface of the first eye.
 6. The system of claim 5,wherein the glint from the surface of the first eye has been generatedfrom a light source attached to a head mounted device.
 7. The system ofclaim 1, wherein the eye measurement data includes one or moredistinctive features of the eye.
 8. The system of claim 1, wherein theeye measurement data includes at least one of facial features and irisdata.
 9. The system of claim 8, wherein the iris data is used asbiometric input data for the generation of a public key and a privatekey, wherein the keys provide identification, authentication, andauthorization for a user to access and control at least one of a localdevice, a wirelessly connected device, and a networked server.
 10. Thesystem of claim 1, wherein the system includes a second sensoroperatively coupled to one or more processors, wherein the second sensoris disposed in a direction the user is facing.
 11. A method forproviding for real-time image control using a dynamically evolvingcognitive architecture based on a natural intent eye movementinterpreter, comprising: an electronic device comprising: one or moreprocessors; and at least one structured illumination device, operativelycoupled to at least one of the processors and disposed towards at leasta first eye of a user; and a first sensor operatively coupled to one ormore processors, wherein the first sensor is disposed towards the atleast first eye of the user; and a non-transitory computer readablememory having a plurality of executable instructions stored thereon, theinstructions, when executed by the one or more processors, cause the oneor more processors to perform operations, comprising: configuring theone or more processors to operate in a first processing mode; andreceiving from the first sensor a first eye measurement data setrepresentative of a first state of the first eye; and processing thereceived first eye measurement data set, wherein the processingcalculates a second state of the first eye, wherein the second state iscalculated to occur at one or more of a future date, future time, futureinterval, future position, future sequence, and future pattern; andresponsive to calculating the second state of the first eye, sending afirst executable instruction to the one or more processors; receivingfrom the first sensor a second eye measurement data set, whereinprocessing the second eye measurement data set calculates a second stateof the first eye; and responsive to calculating the second state of thefirst eye, correlating the predicted second state of the first eye withthe second state of the first eye; and responsive to correlating thepredicted second state of the first eye and the second state of thefirst eye, wherein the correlation result determines the first andsecond eye states are congruent, sending a second executable instructionto the one or more processors.
 12. The method of claim 11, wherein theeye measurement data includes at least one of: eye lid data, eye lashdata, pupil data, cornea data, retina data, iris data, eye movementdata, eye shape data, point of regard data, dwell data, fixation data,saccade data, and illumination data.
 13. The method of claim 11, whereinthe eye signal operations performed by the processor are at least one ofcontinuously, simultaneously, and periodically.
 14. The method of claim11, wherein the first state of the first eye is a first eye gaze. 15.The method of claim 11, wherein the eye measurement data includes thedetection of at least one glint from the surface of the first eye. 16.The method of claim 15, wherein the glint from the surface of the firsteye has been generated from a light source attached to a head mounteddevice.
 17. The method of claim 11, wherein the eye measurement dataincludes one or more distinctive features of the eye.
 18. The method ofclaim 11, wherein the eye measurement data includes at least one offacial features and iris data.
 19. The method of claim 11, wherein theiris data is used as biometric data for the generation of a public keyand a private key, wherein the keys are for the identification andauthentication of the user for access and control of at least one of alocal device, a connected device, a wireless device, and a remoteserver.
 20. The method of claim 11, wherein the system includes a secondsensor operatively coupled to the one or more processors, wherein thesecond sensor is disposed in a direction the user is facing.